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Abstract 

The fundamental property of strongly monotone systems, and strongly cooperative 
systems in particular, is the limit set dichotomy due to Hirsch: if x(0) < y(0), then 
either u>(x) < u(y), or uj(x) = u>(y) and both sets consist of equilibria. We provide 
here a counterexample showing that this property need not hold for (non-strongly) 
cooperative systems. 
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1 Introduction 

The field of cooperative, and more generally monotone systems, provides one of 
the most fruitful areas of theory as well as practical applications -particularly 
in biology- of dynamical systems. For an excellent introduction, see the text- 
book by Smith [4] and the recent exposition [3]. One of its central tools is a 
classical theorem of Hirsch ([1,2]), the "limit set dichotomy" for strongly mono- 
tone (in particular, strongly cooperative) systems, see Theorem 1.16 in [3]. 
The limit set dichotomy states that if x(0) < y(0), then either uj(x) < u>(y), 
or oj(x) = u(y) and both sets consist of equilibria. 

According to the recent survey [3], the problem of deciding if there are any 
cooperative systems for which the dichotomy fails is still open. In [3], example 
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1.24, one finds a system which is monotone but not strongly monotone, for 
which the dichotomy fails. The order in this example is the "ice cream cone" 
order, and the authors explicitly state that it is unknown whether a polyhedral 
cone example exists. A cooperative system is one defined by a set of ordinary 
differential equations x = f(x), where / = (/i, . . . , /„)', with the property 
that §§:(x) > for all i ^ j and all x. Cooperative systems are monotone 
with respect to a polyhedral cone, namely the main orthant in M. n . Thus, a 
counterexample using cooperative systems provides an answer to this open 
question. We provide such a counterexample here. 

To be precise, we construct here two differentiable functions 
f,g :R^R 

such that /(0) = g(0) = 0, xf(x) < and yg(y) < for all x,y ^ 0, and 
consider essentially the following system: 

x = f(x) 

y = g(y) 

z = x + y. 

This system is cooperative. Note that solutions of the x and y equations con- 
verge to zero as t — > oo. Moreover, for this system, there exist x,y such that 
the following property holds: 

There exists some 5 > such that for any solution X with initial condition 
(x(0), y(0), z(0)) T such that \x(0) -x\<6, \y(0) -y\<6, and \z(0)\ < 1, 
the omega-limit set oj{X) is compact and it contains the set 



(o,o,o 



m-\<c<m + \} ■ 

The limit set dichotomy states would imply that, for any initial conditions 

(x(0),y(0),z(0)) T and (x(0), y(0), z(0) f 

for which x(0) < £(0), y(0) < y(0), and z(0) < z(0), and with at least 
one of the inequalities being strict, the corresponding solutions X, X have the 
property that either u(X) = oo{X) or co(X) < uj(X). This last property implies 
in particular that uj(X) and iv(X) are disjoint. Now, for our example, clearly 
uj(X) ^ w(X) as long as z(0) ^ z(0) (since the ^-components of solutions 
are translates by z(0) of the solutions with z(0) = and the cu-limit sets are 
compact), but the omega-sets intersect as long as the x and y initial conditions 
are less than 5, z(0) < z(0), and 

3(0) _ I < i(o) + 1 , |z(0)| <i, |i(o)|<i, 
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i.e. 



< z(0) - z(0) < 1 , \z(0)\ < 1, |i(0)| < 1. 
Thus we contradict the limit set dichotomy. 

In fact, in the above discussion, x and y can be chosen arbitrarily small, that 
is, for any tq > 0, x and y can be chosen so that \x\ < tq, \y\ < tq. Also note 
that by properly choosing a function a(z) and modifying the z-subsystem 
to z = x + y — cr(z), all trajectories of the system can be made to have 
compact closures and the above discussions will still remain valid. For details, 
see Proposition 2.5. 



2 The Example 

To present an example as discussed in Section 1, we first consider the following 
result . 

Lemma 2.1 For any 5 > 0, there exist two C 1 functions p,q : [— l,oo) — > 
[0, oo) such that the following holds: 

(1) both p and q are strictly decreasing functions; 

(2) < p(0) < 5, < q(0) < 5; 

(3) p(t) -> and q(t) -> 0; 

(4) for any a, b G (— 1, 1), the function H a h defined by 

T 



H a , b (T) = J(p(t + a )-q(t + b))dt 
o 



is bounded; and 
(5) for any a, b G (—1, 1), 



limtf a , 6 (T) > 0, liMa,6(T) < 0; 



and 

j^H atb (T)-ljmH a , b (T)>l. 

We will prove the lemma in Section 3.1 by showing that the two functions can 
be chosen as 

P(*) = "^= (1) 

9ij "v^ (t + c ) 3 / 4 ' {) 

where Co can be any number in [a, oo) for some a to be chosen. Observe that 
with proper choices of Co, one can have p(0) = 1/^/cq, q(0) = [1/y/co] + 1/c^ 4 . 
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As a consequence, property (ii) in the lemma can be fulfilled with large enough 
values of c . 

Below we show that, for some C 1 maps / and g, p(t + a) is a solution of 
x = f(x); q(t + a) and —q(t + a) are solutions of y — g(y) for any \a\ < 1. 

It is readily seen that, for \a\ < 1, pit + a) is the solution of the initial value 
problem 

x = — — , x{0) = 



2 ' \/ c o + a ' 

To get a C 1 map g(-) with the desired properties, let ip : (—1, oo) — > (— oo, 0) be 
defined by ip{t) = q'(t), and if : (0,p) — > (—1, oo) be defined by ip(r) = g _1 (r), 
where p = q(—l) > for all t > —1. Let ^(r) = ip o ip(r) for r ^ 0, and 
2(0) = 0. 

Lemma 2.2 The function g : [0,p) — > (— oo,0) is of C 1 . 
We will prove Lemma 2.2 in Section 3.2. 

Extend g from [0,p) to [0, oo) as a C 1 function, and then extend g to R by 
letting g(— r) = — g(r) for r < 0. Still denote the newly extended function by 
g. Then g is a C 1 function. Let /(re) = —x 3 /2. 

Lemma 2.3 For any a,b E (—1, 1), the function (x a (t),y b (t)) T defined by 

x a (t)=p(t + a)), y b (t) = -q(t + b) 
is the solution to the initial value problem 

x = f(x), x a (0) = 

y = g(y), Vb{o) = -q(b). 

The proof of Lemma 2.3 will be given in Section 3.3. 

To get a system as discussed in Section 1, we would like to cascade the system 
(3) with the one- dimensional system i = x + y. To obtain a system for which 
all trajectories are bounded, we choose a C 1 function a : M. — > 1R. with the 
property such that 

• cr(r) = for all |r| < 1 + M, where M = sup{\H ah (t)\ : \a\ < 1, \b\ < l,t> 

o}; 

• ra(r) > for all \r\ > M + 1; and 

• a is proper. 

Consider the system 
x = f(x) 

y = g(y) ( 4 ) 

z = x + y — <j(z). 
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It is clear that the system (4) is cooperative. 
Lemma 2.4 Consider system (4): 

(1) the (x, y)-subsystem is globally asymptotically stable; and 

(2) every trajectory of the system (4) has a compact closure. 

Below we present our final result. We use X(t) = (x(t),y(t), z(t)) T to denote 
a solution of the system (4). 

Proposition 2.5 Consider the cooperative system (4). For any S > 0, there 
exist two trajectories Xx(t) and X 2 {t) with Xi(0) < X 2 (0) and |Xi(0)| < 
5, \X 2 (0)\ < S such that ^ u{X 2 ) and u{Xi) < lo{X 2 ) fails. 

Remark 2.6 In fact, we have obtained a system for which the statement of 
Proposition 2.5 can be made generic in the following sense. For any given Sq > 
0, there exists \X \ < Sq and some 5\ > such that for any pair of trajectories 
X X ,X 2 of the system (4) satisfying |Xi(0) - X \ < S x , \X 2 (0) - X \ < Si, and 
Zl (0) ^ 23(0), |^i (0)| < 1, 1^(0)1 < 1, it holds that lo(Xi) ^ lu(X 2 ) and 
u(Xi) < u{X 2 ) fails. □ 

Proof of Proposition 2.5. Assume that S > is given. Choose c as in (l)-(2) 
large enough so that ^t=t < $ an< ^ l(~ -0 < According to Lemma 2.3, for 
any trajectory of the system 

x = f(x), y = g(y) 

with 

<X (0)<—L=, - q (-l) < y(Q) < - g (l), (5) 



one has x(t) = p(t + a), y(t) = —q(t + b) for some a, b G (— 1, 1), and hence, 

t t 
En / (x(s) + y(s)) ds-M.[ (x(s) + y(s)) ds 

o o (6) 

- ]Eif ai6 (i) - Jim// (*) > 1. 

Let x = 1/y/co, yo = q(0), and 

Then (6) holds for any trajectory (x(t),y(t)) T of (5) with |x(0) — Xo| < 
Si, \y(0) - y \ < Si. Take any \z \ < 1, 



+ y (a:(s) + y(s))ds 



\z + H a , b (t)\ <M + 1, 
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and therefore, Zq + / * (x(s) + y(s)) ds is the solution of the ^-subsystem of (4) 
with the initial value z(0) = Zq (note that o~(s) = when \s\ < M + 1). It then 
follows from statement (v) of Lemma 2.1 that for any x(0), y(0) satisfying (5), 
it holds that 

Eii(z(f)-*(o))>o, ym{z(t)-z(o))<o, 

and 

Jm {z{t) _ z(0 ))] - [JM (*(f) - z(0))] > 1. 
Observe that for any trajectory X(t) of the system (4), 
w(X) D j(0, 0, a) : Um^(t) < a < hmz(t)} . 

For any two solutions := (x(t),y(t), z(t)) and -^(i) := (x(t),y(t), z(t)), 

where x(0) and 7/(0) satisfy (5), and £ (0) 7^ z(0), |z(0)| < 1, |£(0)| < 1, the sets 
of cj-limit points of z(t) and are different (since z(t) — z(t) = z(0) — i(0)). 
Moreover, with z(0) < z(0), 

ITE Z (t) - ljmz(t) = [En (*(*) - 5(0))] - [Jim (*(*) - ,(0)); 

- (^(0) - 5(0)) > 1 - (z(0) - z(0)) > 
if < z(0) - z(0) < 1. Hence, u(X) < u(X) fails. 

3 Proofs of the Lemmas 

In this section, we provide proofs of the results. 
3. 1 Proof of Lemma 2. 1 
First we let 

Po(*) = ^> *W = ^ + ^ I sint 1 / 4 t>\. 
To show that q is decreasing, we consider q' (t): 



(7) 



< 



1 


3 


2t 3 / 2 


4t 7/4 


1 


3 


2t 3 / 2 


4t 7/4 


1 


3 

+ 



8^*1/4 + J_._L_ COS fl/4 



t 3/4 4t 3/4 



4 t 3/2 



(8) 
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1 3 

So, q' (t) < when - > 0. This is the same as t 3/2 < t 7/4 /3, or t 1 / 4 > 
3, that is, t > 81. 

Let p(t) = p (^ + c o)? — <?o(£ + c o), where c > 81 will be chosen later on. 
Now both p and q are differentiable on [0, oo) and monotonically decrease to 
0. For a G (—1, 1) and b G (—1, 1), let H ah {T) be as defined as in Lemma 2.1. 
Then 



H a , b {T) = J (p(t + a) - q(t + b) 



dt 



^t + c + a yjt + c + b (t + c + 6) 3 / 4 



sin(t + c + 6) 1/4 df 



+ c + - \Jt + c + a 1 
V* + c + a V* + c + b ~ (t + c + 6) 3 / 4 



sin(t + c + 6) 1/4 rfl (9) 



Now, for the first term in (9), we have 



\A + c + b - y/t + c + a 



y/t + Co + a\/t + c + b 

b — a 



< 



y/t + cq + ay/t + c + + c + b + \/t + c + a) 
\b — a\ 

V a < 1, \b\ < 1, 



2(t + c - l) 3 / 2 
and hence, the integral 

+ c + - + Co + a 
a/^ + c + a\/t + c + 6 



/ 



dt 



is convergent, and for \a\ < 1, |6| < 1, 

T 



Vi + c + - \A + Co + a 
x/t + c + ay^ + c + b 



dt 



< 



VT > 0. 



(10) 



For the second term in (9), using u = (t + c + b) 1 ^ 4 , one has 



1 



(T+c +6) 1 /4 

sin(t + c + b) 1 ^ dt = j sin u du 



(f + c + 6) 3 / 4 
= - cos(T + c + 6) 1/4 + cos(c + bfl\ 
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Combining this with (10), one sees that, for any \a\ < 1, |6| < 1, H a>b (t) is 
bounded on [0, oo). 

Let c > 82 be of the form (2£;tt + vr/2) 4 . For \b\ < 1, 
Consequently, 

cT-l<{c^b)^< c y^i v|6|<i. 

o o 
This implies that 

_I< cos ( Co + & )i/4<I v|6|<l. 
Thus, 

T 

'"'«/ (/ f f„ - //) :! 1 



lim / / 3/4 sin(t + c + &)V< * = 1 - C os(c + b)^ > \ 



o 

T 



fe / (FT^W sin(< + Co + 6) " 4 * = - 1 " cos(co + 6)V4 " ~\ 

Finally, we let c = (2kir + ir/2) 4 with k large enough so that c > 82 (and 
consequently ^= < 1/4). This way, we get for all \a\ < 1, \b\ < 1, 



UmH ab (T)= lim / [ - - - = ] dt 

T 

-lim /- S i n (t + C +6 )i/4 dt 

T — 7 (t + co + 6) 3 / 4 



00 / 1 1 \ 

where L h — / , = , = 1 (it; and 

a ' b J VV^ + co + a V* + c + & / 

T 

hm# a6 (T)= lim / ( - 1 = - - 1 = ] dt 




T 



-^/(TT7^ si ' l(t + c » + 6)I/4<ft 
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which implies that 

h^H a , b (t)-MkH ajb (t) > I. 

t— *oo 1 v y t— too 1 

Since c was chosen so that \l a ^\ < 1/4, one has 
iiM a , 6 (T) < -1/4, IiM a , 6 (T) > 1/4. 

5. jg Proo/ 0/ Lemma 2. 2 

First of all, it can be calculated (see also (8)) that, for t large enough 



\m\ 



qo(t + c ) 



< 



M 



for some M > 0, where g is defined as in (7). Let tp : (0, p) — > (— l,oo) be 
defined by <p(r) = (? _1 (r), where p = q(— 1) > g(t) for all t > —1. Note that 

lim ip(r) = oo. 

Let g(r) = ip o <p(r) for r ^ 0, and g(0) = 0. Then g is continuous on [0, p), 
and of C 1 on (0,p). Observe that g(r) < for all r e (0, p). Below we show 
that ^ is differentiable at 0. 

Fact 1. There exist some 5 > and some L > such that 



V?(r)>^ Vre (0,5 



;i2) 



To prove Fact 1, write t — q 1 (r). Then r = and 



r > 



(13) 



for some L > and some T > 0. Since q(t) decreases to as t — > 00, it 
follows that for some 5 > 0, it holds that t > j$ for all r e (0, 5 ), this is, 
(12) holds. 
Fact 2. g'(0) = 0. 

Fact 2 follows from Fact 1 combined with (11): 



m<p(r))\ < 



M 



[ v9 ( r )]3/2 



< M 



Lr 



-3/2 



< Mr" 



for all r > in a neighborhood of 0, where M > is some constant. This 
shows that g is differentiable at 0, and g'(0) = 0. 
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Fact 3. q'(r) is continuous at r = 0. 

To prove this fact, we first get an estimate on tp'(t) for t large enough: 



21 



■sint 1 / 4 



16t 10/4— 8f 5/2 

Hence, for some 7\ > and some L\ > 0, 



4t 5/2 16t ll/4 

3 cost l/4 _ 3 cost l/4 _ 1 sint l/4. 



16t 9 / 4 



This implies that, for some <5i > 0, 



IV%(0)I < 



Vre(0,<Ji). 



We also need the following estimate on </ 7 '( r ) near 0: 



(14) 



g(f - c ) 



qo(t) 



> 



2t3/2 4t 7/4 4t 3/2 

Vt > c . 



4^3/2 4t 7/4 

It then can be seen that for some T 2 > and some L 2 > 0, one has 

W(t)\>£k Vt>T 2 , 
which implies that for some 5 2 > 0, 



l?V(0)l > 



L 2 



[ v9 ( r )]3/2 



Vr e (0,5 2 ). 



Finally, 



1^(01 H^'MOMOI 

Ll 1 



^(V(0)"7 



4{<p{r)) 



< 



Li 



as r — > 0. 



Hence, #'(0 ^ s continuous at r = 0. With this we conclude that g is of C 1 on 
[0,p). I 
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3. 3 Proof of Lemma 2. 3 



The statement about x a (t) is certainly clear. 

To treat the part about yb{t), first observe that q'(t) can be written as 
q'(<f(q(t))). Also note that for any |6| < 1, < q(t + b) < q(—l) for all 
t > 0, that is, q(t + b) e (0, p) for all t > 0. Hence, we have 

j t q(t)=g(q(t)), 

that is, q(t) is a solution of the differential equation y = q(y). Let q(t) = —q(t). 
Then 

^r = -f t = -gm) = - 9 (-q(t)) = gW))- 

This shows that —q(t) is also a solution of the equation y = g(y). I 
3.4 Proof of Lemma 2.4 

Since for xf(x) < and yg(y) < for all x 7^ 0,y 7^ 0, both the a> and the 
^/-subsystems are globally asymptotically stable. 

To complete the proof, it is enough to note that every trajectory of the system 
i = -a(z) + h{t) 

is bounded for any choice of bounded function h(t). I 
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